feat(reid): appearance ReID association via standalone reid package - #511
Open
AlexBodner wants to merge 101 commits into
Open
feat(reid): appearance ReID association via standalone reid package#511AlexBodner wants to merge 101 commits into
AlexBodner wants to merge 101 commits into
Conversation
Add the optional trackers[reid] extra (git-pinned roboflow-reid during review) plus the lazy trackers._reid boundary that resolves reid.ReIDModel on demand. Ship association-only, torch-free modules under trackers.core.reid: the ReIDEncoder protocol, FeatureBank (per-track EMA), and appearance_similarity / extract_detection_embeddings. The model stack (encoder, weights, preprocessing, catalog, gallery eval) lives in the standalone reid package and is never vendored. Co-authored-by: Cursor <cursoragent@cursor.com>
Add appearance-IoU fusion (botsort/fusion.py) and wire it into BoT-SORT's
first and unconfirmed association stages, gated by proximity (standard IoU)
and appearance thresholds. Tracklets gain an optional per-track feature
bank; matched high-confidence detections update it. reid_model,
reid_ema_alpha, appearance_threshold, and proximity_threshold are new
BoTSORTTracker params; reid_model is excluded from CLI reflection.
Add --tracker.reid.{enable,model,device,architecture} CLI flags routed
through trackers._reid to reid.ReIDModel.from_pretrained. Importing BoT-SORT
stays torch-free (asserted by an isolation test). Add association/fusion/CLI
unit tests and a reid-backed integration smoke; docs and mkdocs nav cover
the association-only surface and link the model/eval stack out to reid.
CI installs the reid extra (unfrozen while the dep is a git pin).
Co-authored-by: Cursor <cursoragent@cursor.com>
- FeatureBank.update now L2-normalizes the incoming embedding and the resulting EMA, keeping the stored feature on the unit hypersphere (matches upstream BoT-SORT STrack.update_features). reid returns raw embeddings; normalization happens in the bank. Docstrings/tests updated. - Vectorize appearance._l2_normalize_rows. Co-authored-by: Cursor <cursoragent@cursor.com>
AlexBodner
force-pushed
the
feat/core/reid-consume-reid-package
branch
from
July 21, 2026 14:15
5afd3b9 to
fbbe664
Compare
AlexBodner
changed the title
…tion. Ports eval_trackers_reid from the reid-training branch with standalone reid imports and Colab-friendly private git installs for both trackers and re-ID. Co-authored-by: Cursor <cursoragent@cursor.com>
Exclude the default dev group from build/docs sync, authenticate git in integration/pytest workflows, and fix pre-commit failures in the ReID benchmark notebook and codespell config. Co-authored-by: Cursor <cursoragent@cursor.com>
Move the private git dependency to the reid extra only, avoid uv run re-syncing the dev group in build/docs jobs, and fix notebook JSON for pre-commit. Co-authored-by: Cursor <cursoragent@cursor.com>
State what trackers.core.reid covers positively, defer model/eval to the reid package, and drop subtractive architecture prose. Align install extra wording with develop (no roboflow-reid in prose). Co-authored-by: Cursor <cursoragent@cursor.com>
Add MOT17 val with/without ReID reference scores to the ReID API page and align the BoT-SORT optional ReID section with trackers docs style. Co-authored-by: Cursor <cursoragent@cursor.com>
Load frames by MOT index, use per-sequence FPS, re-encode to H.264 for notebook/Colab playback, and embed at the combined panel width. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Inline the optional reid import in the CLI like detection/tune, and keep all BoT-SORT ReID coverage in one test module. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 27 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
notebooks/plot_reid_appearance_distances.py:18
- This example is not runnable:
--dataset bothenters the SoccerNet branch, which exits unless--soccernet-rootis supplied. Include the required root here, as in the SoccerNet-only example.
python notebooks/plot_reid_appearance_distances.py --dataset both
…sume-reid-package Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # docs/evaluations/results.md # docs/guides/reid.md # docs/llms.txt # mkdocs.yml # src/trackers/core/botsort/tracker.py # src/trackers/scripts/track.py # tests/scripts/test_track.py # uv.lock
The CLI rewrite in #413 replaced scripts/track.py with cli/track.py and swapped argparse for jsonargparse over generated dataclasses, so the argparse-based --tracker.reid.* flags had no file left to live in. Appearance loading now travels in a ReIDOptions dataclass registered alongside DetectionOptions, giving --reid.model and friends plus --config YAML support for free. The encoder is built in _init_tracker and injected as the reid_model keyword, mirroring how iou_variant becomes the iou keyword. reid_ema_alpha, appearance_threshold and proximity_threshold now reach the CLI straight from the BoTSORTTracker signature via _tracker_parameter_union, so their flag plumbing is deleted rather than ported. reid_model joins mask_manager in _EXCLUDED_TRACKER_PARAMETERS, which lets the TrackerParameters name check added for it disappear from core/base.py: excluding CLI-invisible constructor args is the CLI's job, and it already had a home for them. Error cases keep their previous messages and exit codes. Selecting a tracker without encoder support is still rejected, but the registry answers that now instead of a hardcoded "botsort" comparison. Co-authored-by: Cursor <cursoragent@cursor.com>
…sume-reid-package Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # README.md
Co-authored-by: Cursor <cursoragent@cursor.com>
… file trackers.core.reid is shared association machinery, not BoT-SORT's, but FeatureBank and appearance_similarity were only covered inside test_botsort_reid.py. The file name implied a coupling the package explicitly denies, and it buried the pure-utility tests among tracker integration ones. Each of the two now sits beside the module it covers, following the flat test_botsort_*.py naming already used in tests/core. What stays behind is genuinely BoT-SORT specific: the import guard, the appearance-IoU fusion gates from botsort/fusion.py, and the tracker integration tests. Co-authored-by: Cursor <cursoragent@cursor.com>
…code The docs tell you to calibrate appearance_threshold on your own footage rather than inherit 0.25 from the paper, then leave you to implement the measurement. The one working implementation sat in a docs-figure script, where nobody who installs trackers can reach it, and getting it right is not obvious: the pairs have to be association-local and the per-sequence quota has to be equal, or one crowded sequence decides the answer. trackers.core.reid.thresholds now owns the sampling, the separability metric and both figures, so the script keeps only what is genuinely dataset-specific, reading MOT-format ground truth and encoding crops (668 to 383 lines). The sampler is a straight move, verified pair-for-pair against the old one on synthetic sequences across four gap bands. Deliberately no suggest_threshold(). A threshold trades same-ID matches against different-ID collisions and only the caller knows which costs them more, so rates_at() reports both sides and the plots mark the thresholds worth comparing, labelled by role — (selected) against (default) — rather than by library, since 0.25 is the default both here and in BoT-SORT. rates_at() accepts a pair at or below the threshold, matching fuse_botsort_reid_association, which drops appearance only once the distance exceeds it. matplotlib joins the reid extra and is imported inside the plot functions, so the tracking path never pays for it. A test pins that. Co-authored-by: Cursor <cursoragent@cursor.com>
The ReID guide explains what appearance association does and shows the numbers, but a reader who wants to trust a threshold on their own footage still has to assemble the whole pipeline: get MOT17, run a detector, run the tracker twice, then measure. This notebook does that end to end in Colab, so the guide's advice to calibrate is something you can execute rather than just read. It merges the two half-answers that existed. The version in roboflow/notebooks had the Colab-first setup and the side-by-side BoT-SORT comparison but no threshold analysis; the one in this repo had the analysis but assumed a local checkout with data already on disk. Detections are computed once and shared between the two tracker runs, so the only difference between them is ReID. The analysis cells call trackers.core.reid.thresholds rather than redefining the sampler, which is why the utility landed first: the notebook stays a thin narrative over library code, and its figures cannot drift from the guide's. Co-authored-by: Cursor <cursoragent@cursor.com>
AlexBodner
force-pushed
the
feat/core/reid-consume-reid-package
branch
from
August 13, 2026 18:03
9a1d848 to
8153c87
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 32 changed files in this pull request and generated no new comments.
Suppressed comments (6)
src/trackers/core/reid/thresholds.py:327
- An empty but shape-valid dataset leaves
indexesempty, after which_draw_distancescalls_split_quota(..., 0)and raisesZeroDivisionErrorinstead of the documentedValueError. Reject zero rows before building the sequence indexes.
unique_sequences = np.unique(sequence_ids)
src/trackers/core/reid/thresholds.py:412
- This catches every
ValueErrorfrom sampling, not only the expected “no pairs in this band” case. Invalid gap bands, mismatched array lengths, and non-finite embeddings are consequently hidden as omitted bands or an empty sweep. Validate shared inputs outside the loop and/or use a dedicated no-pairs exception so genuine input errors still propagate.
except ValueError:
continue
src/trackers/core/reid/thresholds.py:330
- The API describes these values as identity and sequence labels, but coercing them to
intrejects normal string labels and can merge distinct non-integer labels (for example,1.2and1.8). Preserve the original hashable sequence values here and identity values in_SequenceIndexrather than coercing both to integers, or explicitly validate and document integer-only inputs.
int(sequence): _SequenceIndex(np.flatnonzero(sequence_ids == sequence), ids, frame_ids)
for sequence in unique_sequences
src/trackers/cli/track.py:746
architectureis documented as requiringmodel, but this predicate ignores it. Passing only--reid.architecturetherefore bypasses_load_reid_model, silently runs geometry-only tracking, and never raises the existing validation error. Treat a non-Nonearchitecture as a ReID request so that validation executes.
return reid.enable or reid.model is not None
docs/cookbooks/how-to-add-reid-to-trackers.ipynb:226
- For a zero-based detection file,
offsetis-1, butmax(offset, 0)turns it back into0; frame 0 therefore remains frame 0 and never matches the tracking loop, which starts at 1. Subtract the signed offset directly so both zero- and one-based files align to frame 1.
" by_frame.setdefault(frame - max(offset, 0), []).append([x1, y1, x2, y2, score])\n",
src/trackers/core/reid/thresholds.py:208
pairable_idsonly means an identity has multiple crops overall, not that it has a valid partner in the requested gap band. When a selected identity/anchor has no partner, the whole draw is rejected and another identity is selected, so accepted samples are weighted by each identity’s success rate; dense tracks can dominate despite the documented uniform-per-identity sampling. Build the candidate identities and anchors for the active gap band first, then sample uniformly from valid candidates.
This issue also appears in the following locations of the same file:
- line 327
- line 329
- line 411
if not index.pairable_ids:
return None
slots = index.slots_by_id[index.pairable_ids[int(rng.integers(len(index.pairable_ids)))]]
anchor = int(slots[int(rng.integers(len(slots)))])
window = index.window(index.frames[slots], int(index.frames[anchor]), minimum_frame_gap, maximum_frame_gap)
partner_slot = _pick_in_window(rng, window)
…cept, fix _reid_requested, preserve import errors [resolve group] PR #511 — items 9, 13, 14, 16, 32 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
…rmed-track fusion [resolve group] PR #511 — items 22, 23, 24 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
[resolve group] PR #511 — item 25 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
[resolve group] PR #511 — items 12, 17, 26, 27, 28 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
[resolve group] PR #511 — item 31 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
…n histogram xlim [resolve group] PR #511 — items 15, 18, 33 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
Add extract_ground_truth_embeddings beside extract_detection_embeddings. It walks {sequence}/gt/gt.txt against img1, drops ignore-flagged rows, optionally keeps MOT classes, and renumbers identities across sequences.
Re-export it from trackers.core.reid. Delete notebooks/plot_reid_appearance_distances.py and point the cookbook, API page, and guide at the loader.
Update the MOT17 and SoccerNet gap-sweep PNGs. Align the MOT17 rate table (1.6%, 31.8%, 28.7%) and the SoccerNet different-ID range (44% to 51%).
Co-authored-by: Cursor <cursoragent@cursor.com>
---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…iden reid docstring scope [resolve group] PR #511 — items 19, 20, 29, 30 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
…age' into feat/core/reid-consume-reid-package
[resolve group] PR #511 — Step 9 lint/QA gate --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…com/roboflow/trackers into feat/core/reid-consume-reid-package # Conflicts: # notebooks/plot_reid_appearance_distances.py # tests/core/test_reid_appearance.py
Fusion is numpy-only association math over track-detection similarity matrices, so it belongs in the shared ReID association layer next to appearance_similarity and FeatureBank rather than under core/botsort. This makes fusion methods reusable by any tracker. Pure relocation: no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
thresholds.py was hard to follow for its size. Same behaviour throughout — sampled distances verified identical to the previous implementation across 4 seeds x 7 gap bands (224k distances) and across 15 identity-label dtypes (ints, floats, float32, strings, bytes, bools, datetime64, object arrays, None and NaN). - roc_auc: replace the searchsorted rank arithmetic with the pairwise definition it documents. Note the O(n*m) scaling inline. - collapse the same-ID/different-ID sampling helpers into _SequenceIndex.get_candidates / get_anchor_groups, dropping the callback dispatcher, its TypeVar and two type aliases. - fold the four window helpers into one method returning positions, which removes the slot/position duality the comments used to explain. - index tracks by position (tracks / track_of) rather than looking a track up by its identity label while drawing. Labels are hashed once, when the index is built, so labels that never compare equal to themselves — NaN from a pandas column with missing ids — keep working. Covered by a test. - drop _scalar_key. Its job was to stop int(identity) merging non-int labels; using the label directly does that, and unhashable labels still raise TypeError from the dict itself. - inline the matplotlib import at both plot functions instead of a module returning _pyplot() helper, matching core/masks/sam.py and cutie.py. - name things the way the rest of core/ does: "candidate" over the invented "partner", get_* for methods that compute a collection. - drop the dead maximum_attempts_per_pair parameter; make the sweep's percentile band a real parameter instead of a module constant. - remove comments that restated their own docstrings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds optional appearance-ReID to BoT-SORT, sourcing the model stack from the standalone
reidpackage. Trackers ships only NumPy association glue (ReIDEncoder,FeatureBank,appearance_similarity,extract_detection_embeddings) plus BoT-SORT appearance–IoU fusion. Encoder weights, preprocessing, catalog, and gallery evaluation live inreidand are never vendored.This branch is merged with current
develop(including McByte) and is independent of the earlier stacked ReID PRs (#480/#501/#502).Added things
trackers[reid]optional extra →reid>=0.1.0.dev0,<0.2(PyPI).reid.*in mypy overrides;uv.lockupdated.src/trackers/core/reid/, torch-free):ReIDEncoder,FeatureBank(BoT-SORT EMA L2 before/after blend),appearance_similarity,extract_detection_embeddings.reid.ReIDModelat use time (same pattern as detection); importing BoT-SORT stays free ofreid/ torch (isolation test).reid_model,reid_ema_alpha,appearance_threshold,proximity_threshold.--tracker.reid.{enable,model,device,architecture}ontrackers track(BoT-SORT only).--extra reidrequired),@integrationsmoke with a realreid.ReIDModel, isolation test.docs/learn/reid.md(threshold guide + MOT17/SoccerNet association-local plots),docs/api/reid.md, install notes, BoT-SORT Appearance ReID section;notebooks/eval_trackers_reid.ipynbinstalls from PyPI.ci-tests.yml) syncs without--extra reid; integration (ci-integrations.yml) keeps--extra reid.Pre-merge checklist
reidto PyPI (0.1.0.dev0).pyproject.tomlto a PyPI range (reid>=0.1.0.dev0,<0.2) and refreshuv.lock.develop(McByte / mask extras) so PR CI can run.--extra reid(smoke should run, not skip).